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Description 

Background of the Invention 

The invention relates to an apparatus for predicting 
and evading potential crash of a vehicle and for predict- 
ing real crash of a vehicle and acting correspondingly 
according to the preamble of claim 1 . Furthermore, the 
invention relates to a method according to the preamble 
of claim 9. 

A driver has an unconscious and immediate sense 
of various conditions through the objects in view and, as 
a case may be, he must take an action to evade any 
possible crash or collision. However, drivers will often be 
panicked at the emergency of above their sense. Such 
a panicked driver may sometimes be the last one who 
can cope with the emergency to ensure the active safety 
of the vehicle. Besides, the response delay to stimuli in 
varying degrees is inherent to human beings, so that it 
is impossible in some cases to evade crash or danger 
by physical considerations. With this respect, various 
techniques have been developed to evade collision by 
means of mounting on a vehicle a system for determin- 
ing the possibility of crash in a mechanical or electrical 
manner before it happens. Accidents could be reduced 
if drivers had an automatic system or the like warning of 
potential collision situations. 

An apparatus comprising a system for predicting ' 
and evading crash of a vehicle including image pick-up 
means, crash predicting means, safety drive ensuring 
means and a neural network according to the preamble 
of claim 1 is known from JP-A-4008639 (Abstract). 

In WO-A-90/02985 a system for predicting and 
evading crash of a vehicle comprising a sensor system 
mounted on the vehicle for observing the surrounding in 
a running direction is disclosed. A crash predicting sys- 
tem is successively supplied with actual sensor system 
data for predicting occurence of crash between the vehi- 
cle and potential dangerous objects on the roadway and 
for producing an operational signal when there is possi- 
bility of crash. In response to the operational signal the 
vehicle is operated to evade the crash. 

Another moveable device with supersonic sensors 
and a neural network to further process the data taken 
by the supersonic sensor system is disclosed in DE-A- 
4001493. Furthermore, in this moveable device learning 
data are used to train the neural network. 

In general, neural networks are known from "Paral- 
lel distributed Processing", Vol. 1 : Foundations, A Brad- 
ford Book, The MIT Press, Cambridge, Mass., London, 
England, by David E. Rumelhart, James L. McClelland 
and the PDP Research Group. 

An automobile collision avoidance radar is typically 
used as this automatic system. Such an automobile col- 
lision avoidance radar is disclosed in, for example, M. 
Kiyoto and A. Tachibana, Nissan Technical Review: 
Automobile Collision-Avoidance Radar, Vol. 18, Dec. 
1982 that is incorporated by reference herein in its 



entirety. The radar disclosed comprises a small radar 
radiation element and antennas installed at the front 
end of a vehicle. A transmitter transmits microwaves 
through the radiation element towards the headway. 

5 The microwave backscatter from a leading vehicle or 
any other objects as echo returns. The echo returns are 
received by a receiver through the antennas and sup- 
plied to a signal processor. The signal processor carries 
out signal processing operation to calculate a relative 

70 velocity and a relative distance between the object and 
the vehicle. The relative velocity and the relative dis- 
tance are compared with predetermined values, respec- 
tively, to determine if the vehicle is going to collide with 
the object. The high possibility of collision results in acti- 

is vation of a proper safety system or systems. 

However, the above mentioned radar system has a 
disadvantage of faulty operation or malfunctions, espe- 
cially when the vehicle implementing this system 
passes by a sharp curve in a road. The radar essentially 

20 detects objects in front of the vehicle on which it is 
mounted. The system thus tends to incorrectly identify 
objects alongside the road such as a roadside, guard 
rails or even an automobile correctly running on the 
adjacent lane. 

25 An intelligent vehicle has also been proposed that 
comprises an image processing system for cruise and 
traction controls. Ever-changing views spreading ahead 
the vehicle are successively picked up as image pat- 
terns. These image patterns are subjected to pattern 

30 matching with predetermined reference patterns. The 
reference patterns are classified into some categories 
associated with possible driving conditions. For exam- 
ple, three categories are defined for [straight running, 
right turn and left turn. When a matching result indicates 

35 the presence of potentially dangerous objects in the 
picked up image, a steering wheel and a brake system 
are automatically operated through a particular mecha- 
nism to avoid or evade crash to that object. 

The image processing system of the type described 

40 is useful in normal driving conditions where the pattern 
matching can be effectively made between the image 
patterns successively picked up and the reference pat- 
terns for safety driving control. However, image patterns 
representing various conditions on the roadway should 

45 be stored previously in the intelligent vehicle as the ref- 
erence patterns. Vehicle orientation at initiation of crash 
varies greatly, so that huge numbers of reference pat- 
terns are required for the positive operation. This means 
that only a time-consuming calculation will result in a 

so correct matching of the patterns, which is not suitable 
for evading an unexpected crash. 

It is, of course, possible to increase operational 
speed of the pattern matching by using a large dedi- 
cated image processor. However, such a dedicated 

55 processor is generally complex in structure and rela- 
tively expensive, so that it is difficult to apply the same 
as the on -vehicle equipment. In addition, on-vehicle 
image processors, if achieved, will perform its function 
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sufficiently only in the limited applications such as a 
supplemental navigation system during the normal 
cruising. 

Summary of the Invention s 

An object of the present invention is to provide a 
system for predicting and evading crash of a vehicle 
using neural networks. 

Another object of the present invention is to provide 
a system capable of training neural networks by means 
of collecting image data representing ever-changing vis- 
tas along the travel direction of a vehicle until the vehicle 
collides with something. 

It is yet another object of the present invention to 
provide a system for predicting crash though matching 
operation between data obtained on driving a vehicle 
and data learned by neural networks. It is still another 
object of the present invention to provide a system for 
evading crash of a vehicle using neural networks to 
actuate a vehicle safety system for protecting an occu- 
pant. 

In order to achieve the above-mentioned objects, 
an apparatus comprising the features of claim 1 and a 
method comprising the features of claim 9 are provided. 

The neural network comprises at least an input 
layer and an output layer, and the training data are sup- 
plied to the input layer while the output layer is supplied 
with, as teacher data, flags representing expected and 
unexpected crash, respectively, of the vehicle. In addi- 
tion, the neural network may comprise a two-dimen- 
sional self-organizing competitive learning layer as an 
intermediate layer. 

Other advantages and features of the present 
invention will be described in detail in the following pre- 
ferred embodiments thereof. 

Brief Description of the Drawings 

Fig. 1 is a block diagramm of a conventional system 
for predicting and evading crash of a vehicle; 
Fig. 2 is a schematic view showing a processing 
element in a typical neural network; 
Fig. 3 is a graphical representation of a sigmoid 
function used as a transfer function for training neu- 
ral networks; 

Fig. 4 is a block diagram of a system for predicting 
and evading crash of a vehicle using neural net- 
works according to the first embodiment of the 
present invention; 

Fig. 5(a) is a schematic structurai diagram of a 
crash predicting circuit in Fig. 4 realized by a neural 
network of three layers; 

Fig. 5(b) shows an example of an input layer con- 
sisting of a two-dimensional array of processing 
elements of the neural network shown in Fig. 5(a); 
Figs. 6(a) and 6(b) are exemplified views picked up, 
as the training image data supplied to the neural 



network, at different time instances during driving 
an experimental vehicle; 

Fig. 7 is a view showing an example of an image 
data obtained during driving a utility vehicle; 
Fig. 8 is a view showing another example of an 
image data obtained during driving a utility vehicle; 
and 

Fig. 9 is a block diagram of a system for predicting 
and evading crash using neural networks according 
to the second embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

A conventional system for predicting and evading 
crash of a vehicle is described first to facilitate an under- 
standing of the present invention. Throughout the fol- 
lowing detailed description, similar reference numerals 
refer to similar elements in all figures of the drawing. 

In the following description, the term "crash" is used 
in a wider sense that relates to ail unexpected traffic 
accidents. Accidents other than crash include a turno- 
ver or fall of a vehicle, with which the phenomenon of 
"crash" is associated in some degrees therefore the use 
of term crash as a cause of traffic accidents. 

As shown in Fig. 1 , an image pick-up device 21 is 
mounted at a front portion of an automobile 10 to pick 
up ever-changing images as analog image data. This 
image pick-up device 21 is any one of suitable devices 
such as a charge-coupled-device (CCD) camera. The 
image data are subject to sampling for a sampling range 
A T during a predetermined sampling period A t. The 
image data are collected up to crash. In this event, the 
image pick-up range of the image pick-up device 21 cor- 
responds to a field of view observed through naked 
eyes. 

The image pick-up device 21 is connected to an 
input interface 22. The analog image data obtained by 
the image pick-up device 21 are supplied to the input 
interface 22. The input interface 22 serves as an ana- 
log-to-digital converter for converting the analog image 
data into digital image data. More particularly, the 
picked up images are digitized by means of dividing the 
same into tiny pixels (data elements) isolated by grids. It 
is preferable to eliminate noises and distortions at this 
stage. The input interface 22 is also connected to a 
speed sensor 23, a steering gear ratio sensor 24 and a 
signal processor 30. The speed sensor 23 supplies 
velocity data to the signal processor 30 through the 
input interface 22. The velocity data represents an 
actual velocity of the automobile 10 at the time instant 
when the image pick-up device 21 picks up an image of 
a view. Likewise, the steering gear ratio sensor 24 sup- 
plies steering gear ratio data to the signal processor 30 
through the input interface 22. The steering gear ratio 
data represents an actual steering gear ratio of the 
automobile 10. 

The signal processor 30 comprises a central 
processing unit (CPU) 31, a read-only memory (ROM) 
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32 and a random-access memory (RAM) 33. CPU 31. 
ROM 32 and RAM 33 are operatively connected to each 
other through a data bus 34. To evade potentially dan- 
gerous objects. CPU 31 carries out calculation opera- 
tion in response to the image, velocity and steering gear 5 
ratio data given through the input interlace 22. CPU 31 
performs proper functions according to programs stored 
in ROM 32 and RAM 33. The outputs of the signal proc- 
essor 30 is transmitted through an output interface 40. 
ROM 32 stores a table relating to numerical values 10 
required for the calculation. It also stores a table repre- 
senting operational amount for a safety drive ensuring 
arrangement 50. On the other hand, RAM 33 stores pro- 
grams for use in calculating an . optimum operational 
amount for the safety drive ensuring arrangement 50. A 15 
program for this purpose is disclosed in, for example, 
Teruo Yatabe, Automation Technique: Intelligent Vehi- 
cle, pages 22-28. 

The signal processor 30 first determines, according 
to the picked up image data, whether there is a space 20 
available on the roadway to pass through. When there is 
enough space to pass through and a potentially danger- 
ous object is present on the roadway, the signal proces- 
sor 30 calculates optimum operational amount for the 
safety drive ensuring arrangement 50 to operate the 25 
same. In Fig. 1, the safety drive ensuring arrangement 
50 consists of a steering actuator 51 , a throttle actuator 
52 and a brake actuator 53. If the signal processor 30 
determines that it is necessary to operate these actua- 
tors, it produces steering gear ratio command, set 30 
velocity command, and brake operation command. The 
steering actuator 51, the throttle actuator 52 and the 
brake actuator 53 are operated depending on the condi- 
tion in response to the steering gear ratio command, the 
set velocity command and the brake operation com- 35 
mand, respectively. 

The actuators are for use in actuating occupant pro- 
tecting mechanism such as a brake device. Operation of 
these actuators is described now. 

The steering actuator 51 is a hydraulic actuator for 40 
use in rotating steering wheel (not shown) in an emer- 
gency. In this event, the steering wheel is automatically 
rotated according to the steering gear ratio and rota- 
tional direction indicated by the steering gear ratio com- 
mand. The operational amount of the steering or 45 
hydraulic actuator can be controlled in a well-known 
manner through a servo valve and a hydraulic pump, 
both of which are not shown in the figure. 

The throttle actuator 52 acts to adjust opening 
amount of a throttle valve (not shown) to decrease so 
speed while evading objects or so on. 

The brake actuator 53 performs a function to grad- 
ually decrease speed of a vehicle in response to the 
brake operational command. The brake actuator 53 is 
also capable of achieving sudden brake operation, if 55 
necessary. 

As mentioned above, CPU 31 carries out its opera- 
tion with the tables and programs stored In ROM 32 and 
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RAM 33, respectively, for every one picked up image 
data. The conventional system is thus disadvantageous 
in that the calculation operation requires relatively long 
time interval as mentioned in the preamble of the instant 
specification. 

On the contrary, a system according to the present 
invention uses image data representing ever-changing 
views picked up from a vehicle until it suffers from an 
accident. These image data are used for training a neu- 
ral network implemented in the present system. After 
completion of the training/the neural network is imple-. 
mented in a utility vehicle and serves as a decision mak- 
ing circuit for starting safety driving arrangements to 
evade crash, which otherwise will certainly be hap- 
pened, the neural network predicts crash and evades 
the same by means of properly starting an automatic 
steering system or a brake system. 

A well-known neural network is described first to 
facilitate an understanding of the present invention and, 
following which preferred embodiments of the present 
invention will be described with reference to the draw- 
ing. 

A neural network is the technological discipline con- 
cerned with information processing system, which has 
been developed and still in their development stage. 
Such artificial neural network structure is based on our 
present understanding of biological nervous systems. 
The artificial neural network is a parallel, distributed 
information processing structure consisting of process- 
ing elements interconnected unidirectional signal chan- 
nels called connections. Each processing element has 
a single output connection that branches into as many 
collateral connections as desired. 

A basic function of the processing elements is 
described below. 

As shown in Fig. 2. each processing element can 
receive any number of incoming functions while it has a 
single output connection that can be fan out into copies 
to form multiple output connections. Thus the artificial 
neural network is by far more simple than the networks 
in a human brain. Each of the input data x1 , x2, , xi is 
multiplied by its corresponding weight coefficient w1, 
w2, , wi, respectively, and the processing element 
sums the weighted inputs and passes the result through 
a nonlinearity. Each processing element is character- 
ized by an internal threshold or offset and by the type of 
nonlinearity and processes a predetermined transfer 
function to produce an output f(X) corresponding to the 
sum (X ■ E xi • wi ). In Fig. 2, xi represents an output of 
an i-th processing element in an (s-1)-th layer and wi. 
represents a connection strength or the weight from the 
(s-1)-th layer to the s-th layer. The output f(X) repre- 
sents energy condition of each processing element. 
Though the neural networks come in a variety of forms, 
they can be generally classified into feedforward and 
recurrent classes. In the latter, the output of each 
processing element is fed back to other processing ele- 
ments via weights. As described above, the network has 
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an energy or an energy function associated with it that 
will be minimum finally. In other words, the network is 
considered to have converged and stabilized when out- 
puts no longer change on successive iteration. Means 
to stabilize the network depends on the algorithm used. 5 

The back propagation neural network is one of the 
most important and common neural network architec- 
ture, which is applied to the present invention. In this 
embodiment, the neural network is used to determine if 
there is a possibility of crash. When the neural network 10 
detects the possibility of crash, it supplies an opera- 
tional command to a safety ensuring unit in a manner 
described below. As well known in the art, the back 
propagation neural network is a hierarchical design con- 
sisting of fully interconnected layers of processing ele- is 
ments. More particularly, the network architecture 
comprises at least an input layer and an output layer. 
The network architecture may further comprise addi- 
tional layer or N hidden layers between the input layer 
and the output layer where N represents an integer that 20 
is equal to or larger than zero. Each layer consists of 
one or more processing elements that are connected by 
links with variable weights. The net is trained by initially 
selecting small random weights and internal thresholds 
and then presenting all training data repeatedly. 25 
Weights are adjusted after every trial using information 
specifying the correct result until weights converge to an 
acceptable value. The neural network is thus trained to 
automatically generate and produce a desired output for 
an unknown input. 30 

Basic learning operation of the back propagation 
neural network is as follows. First, input values are sup- 
plied to the neural network as the training data to pro- 
duce output values, each of which is compared with a 
correct or desired output value (teacher data) to obtain 35 
information indicating a difference between the actual 
and desired outputs. The neural network adjusts the 
weights to reduce the difference between them. More 
particularly, the difference can be represented by a well- 
known mean square error. During training operation, the 40 
network adjusts all weights to minimize a cost function 
equal to the mean square error. Adjustment of the 
weights is achieved by means of back propagating the 
error from the output layer to the input layer. This proc- 
ess is continued until the network reaches a satisfactory 45 
level of performance. The neural network trained in the 
above mentioned manner can produce output data 
based on the input data even for an unknown input pat- 
tern. 

The generalized delta rule derived with the steepest so 
descent may be used to optimize the learning proce- 
dure that involves the presentation of a set of pairs of 
input and output patterns. The system first uses the 
input data to produce its own output data and then com- 
pares this with the desired output. If there is no differ- 55 
ence, no learning takes place and otherwise the weights 
are changed to reduce the difference. As a result of this 
it becomes possible to converge the network after a rel- 



atively short cycle of training. 

To train the net weights on connections are first ini- 
tialised randomly and input data (training data) are suc- 
cessively supplied to the processing elements in the 
input layer, Each processing element is fully connected 
to other processing elements in the next layer where a 
predetermined calculation operation is carried out In 
other words, the training input is fed through to the out- 
put. At the output layer the error is found using; for 
example, a sigmoid function and is propagated back to 
modify the weight on a connection. The goal is to mini- 
mize the error so that the weights are repeatedly 
adjusted and updated until the network reaches a satis- 
factory level of performance. A graphical representation 
of sigmoid functions is shown in Fig. 3. 

In this embodiment a sigmoid function as shown in 
Fig. 3 is applied as the transfer function for the network. 
The sigmoid function is a bounded differentiate real 
function that is defined for all real input values and that 
has a positive derivative everywhere. The central por- 
tion of the sigmoid (whether it is near 0 or displaced) is 
assumed to be roughly linear. With the sigmoid function 
it becomes possible to establish effective neural net- 
work models. 

As a sigmoid function parameter in each layer, a y- 
directional scale and a y-coordinate offset are defined. 
The y-directional scale is defined for each layer to 
exhibit exponential variation. This results in improved 
convergence efficiency of the network. 

It is readily understood that other functions may be 
used as the transfer function. For example, in a sinusoi- 
dal function a differential coefficient for the input sum in 
each processing element is within a range equal to that 
for the original function. To use the sinusoidal function 
results in extremely high convergence of training though 
the hardware for implementing the network may be 
rather complex in structure. 

An embodiment of the present invention is 
described with reference to Figs. 4 through 9. 

Fig. 4 is a block diagram of a system for predicting 
and evading crash of a vehicle using neural networks 
according to the first embodiment of the present inven- 
tion. A system in Fig. 4 is similar in structure and opera- 
tion to that illustrated in Fig. 1 other than a crash 
predicting circuit 60. Description of the similar compo- 
nents will thus be omitted by the consideration of evad- 
ing redundancy. Fig. 5 is a schematic structural diagram 
of the crash predicting circuit 60 illustrated in Fig. 4 real- 
ized by a neural network of three layers. 

The crash predicting circuit 60 in this embodiment 
is implemented by a neural network architecture of a 
hierarchical design with three layers as shown in Fig. 
5(a). The input layer 61 consists of n processing ele- 
ments 61-1 through 61 -n arranged in parallel as a one- 
dimensional linear form. Each processing element in 
the input layer 61 is fully connected in series to the 
processing elements in a hidden layer 62 of the net- 
work. The hidden layer 62 is connected to an output 
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layer 63 of a single processing element to produce an 
operational command described below. Fig. 5(b) shows 
an input layer consisting of a two-dimensional array of 
processing elements. In this event, the image data are 
supplied to the input layer as a two-dimensional data 
matrix of n divisions. Basically, the Input and the hidden 
layers can have any geometrical form desired. With the 
two-dimensional array, the processing elements of each 
layer may share the same transfer function, and be 
updated together. At any rate, it should be considered 
that each processing element is fully interconnected to 
the other processing elements in the next layer though 
only a part of which are shown in Fig. 5(a) to evade 
complexity. 

Referring now to Fig. 6 in addition to Fig. 5, illus- 
trated are views picked up, as the image data for use in 
training the neural network. The image pick-up device 
21 picks up ever-changing images as analog image 
data as described above in conjunction with the conven- 
tional system. This image pick-up device 21 is also any 
one of suitable devices such as a CCD camera. The 
image pick-up operation is carried out during running of 
a vehicle at higher speed than a predetermined one. 
The image data are subject to sampling for a sampling 
range A T during a predetermined sampling period A t. 
The image data are collected before and just after 
pseudo crash. The image pick-up range of the image 
pick-up device 21 corresponds to a field of view 
observed through naked eyes. A view shown in Fig. 6(a) 
is picked up when a station wagon (estate car) 80a on 
the opposite lane comes across the center line. A view 
shown in Fig. 6(b) is picked up when an automobile 80b 
suddenly appears from a blind corner of a cross-street. 
These ever-changing images are collected as the train- 
ing data for the neural network. 

The image data effectively used for the crash eva- 
sive purpose are those which allow continuous recogni- 
tion of the ever-changing views before and just after 
pseudo crash. With this respect, the image pick-up 
device 21 picks up the images of a vehicle or other 
obstructions located at a relatively short headway. In 
addition, the picked up images preferably are distinct 
reflections of the outside views. 

The data elements consisting of one image are 
simultaneously supplied to the input layer 61 in parallel. 
In other words, each data element is supplied to the 
respective processing element of the input layer 61 . The 
digital image data may be normalized before being sup- 
plied to the input layer 61 to increase a data processing 
speed. However, each processing element of the input 
layer 61 essentially receives the data element obtained 
by dividing the image data previously. The data ele- 
ments are subjected to feature extraction when supplied 
to the hidden layer 62. 

In typical image processing, feature extraction is 
carried out according to any one of various methods of 
pattern recognition to clearly identify shapes, forms or 
configurations of images. The feature-extracted data 



are quantized for facilitate subsequent calculations. In 
this event, separate analytical procedure is used for 
region partitioning or for extraction of configuration 
strokes. In other words, a particular program is neces- 

5 sary for each unit operation such as region partitioning, 
feature extraction, vectorization and so on. Compared 
with this, the prediction system according to the present 
invention requires no program based on each operation 
or procedure because a unique algorithm is established 

10 on completion of network training. This single algorithm 
allows, to perform necessary functions without using 
separate algorithms or programs. 

In a preferred embodiment, the feature extraction is 
directed to the configuration of an object defining the 

75 driving lanes such as shoulders, curbs, guard rails or 
the center line. The feature may also be extracted on 
regions such as carriageways. The neural network 
learns these configurations and regions during training 
process. This process is continued until the network 

20 reaches a satisfactory level of performance, the neural 
network is thus trained while carrying out feature extrac- 
tion on the input image. Weights are adjusted after 
every trial on the quantized image data, so that the lat- 
est training data is weighted according to the latest 

25 result of adjustment and then supplied to the hidden 
layer 62. In addition, the neural network can be trained 
with image data including an object at time-varying posi- 
tions. In this event, any one of suitable methods may be 
used for digital image processing. 

30 In the present embodiment, each digital data indic- 
ative of ever-changing view at a certain sampling time 
instance is divided into n data elements. A product of n 
represents a positive integer which is equal in number 
to the processing elements in the input layer 61 . In other 

35 words, the series of time sequential data is picked up as 
continuous n data elements to be supplied in parallel to 
the n by m processing elements in the input layer 61 as 
the training data. At the same time, an operational sig- 
nal is supplied to the output layer 63 of the network as 

40 teacher data. The operational signal may be a logic "1" 
for representing crash of the automobile 10 after elapse 
of a predetermined time interval from the sampling time 
instant corresponding to the image data just having 
been, supplied to the input layer 61 . 

45 In the same manner, the picked up image data and 
its corresponding teacher data are successively sup- 
plied to the crash predicting circuit 60. The crash pre- 
dicting circuit 60 is continuously trained until the 
network reaches a satisfactory level of performance. 

so After completion of training, the network is capable of 
matching the picked up image with the possibility of 
crash. The accuracy of prediction is improved by means 
of supplying images for a case of "safe" state to the neu- 
ral network on learning. 

55 The neural network thus learns the relative position 
between the vehicle on which it is mounted and objects 
at a short headway. As a result of this learning, the 
crash predicting circuit 60 enables to prediction of crash 
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expected to be happened a few seconds later according 
to this relative position. While outside views change 
every moment and a vehicle in practice encounters var- 
ious objects and situations, a series of repeated training 
can yield stereotyped data patterns. 5 

The neural network program that has already been 
trained can be memorized in a read only memory 
(ROM) as an application. In this event the network pro- 
gram is memorized after being compiled and translated 
into a machine language. The ROM is implemented in a to 
predetermined IC chip or the like as an inherent circuit. 
The IC chip is mounted on a circuit for the air bag sys- 
tem in an automobile. 

As mentioned above, the crash predicting circuit 60 
supplies the operational signal to the safety drive ensur- 15 
ing arrangement 50 when it predicts occurrence of 
crash. In response to this operational signal the safety 
drive ensuring arrangement 50 can perform proper 
function to evade crash. 

For more clear understanding of the present inven- 20 
tion, two cases are described those results in "safe" 
state of the automobile 80c. 80d. Fig. 7 shows an exem- 
plified image including an oncoming vehicle 80c running 
on the opposite lane. The situation being far from dan- 
ger as shown in Fig. 7 may allow the system of the 25 
present invention to bypass the crash predicting circuit 
60. Alternatively, the crash predicting circuit 60 may pro- 
duce an operational signal of logic "0" to represent this 
"safe" condition. 

A view shown in Fig. 8 represents a situation when 30 
a vehicle 80d on the opposite lane comes across the 
center line in the far distance ahead. The vehicle 80d is 
going to return to the lane where it ought to be. The sub- 
sequent image data indicate that the oncoming vehicle 
80d takes an action to evade crash. In other words, the 35 
oncoming vehicle 80d is expected to return to the 
proper lane before the vehicle mounting the crash pre- 
dicting circuit 60 passes by the vehicle 80d. Accordingly, 
the crash predicting circuit 60 determines that there are 
no hazardous objects ahead. 40 

If a vehicle on the opposite lane comes across the 
center line or a vehicle suddenly appears from a blind 
corner of a cross-street as shown in Figs. 5(a) and 5(b), 
the crash predicting circuit 60 carries out prediction 
operation in accordance with the image data showing 45 
these situations. Expected hazards make the crash pre- 
dicting circuit 60 actuate the safety drive ensuring 
arrangement 50 in the manner described above. 

Another embodiment of the present invention will 
be described below in which the neural network com- so 
prises an intermediate la^er having a self -organization 
function and a competitive learning function to positively 
respond to various unknown data with less training 
data. As well known in the art, in the serf-organization a 
network modifies itself in response to inputs. Examples ss 
of the use of self-organizing training include the compet- 
itive learning law applied to the present embodiment. 

As shown in Fig. 9 the neural network according to 



this embodiment comprises a two-dimensional self- 
organized competitive learning layer 64 interposed 
between the input layer 61 and the hidden layer 62. The 
two-dimensional self-organized competitive learning 
layer 64 is referred as to the two-dimensional Kohonen 
layers (2D-K layer) which in this embodiment comprises 
p by q layers consisting of a two-dimensional array of 
processing elements. The input layer 61 may consists of 
either one or two-dimensional array of processing ele- 
ments. The 2D-K layer 64 can have any geometrical 
form desired. In this embodiment, it is also considered 
that each processing element is fully interconnected to 
the other processing elements in the next layer though 
only a part of which are shown in Fig. 9 to evade com- 
plexity. 

The processing elements in the 2D-K layer 64 com- 
pete with one another to determine the "winner" on the 
basis of minimum distance. More particularly, a prede- 
termined distance can be obtained by, in this embodi- 
ment, n processing elements for each set of the input 
data. The similarity for each of the n input data corre- 
sponds to the distance to select similar combination of 
processing elements. The selected processing ele- 
ments becomes "winner" for facilitating determination 
on attributes of unknown data. 

More particularly, the winning three Kohonen's 
processing elements are determined among the fourth 
processing elements to supply output data. Unknown 
data are preprocessed on the basis of classification for 
the input data due to the self-organization on learning. 
The output value thereof is supplied to the subsequent 
hidden layer. 

With an additional normalization layer 65 may be 
interposed between the input layer 61 and the 2D-K 
layer 64 as shown in Fig. 9. With this normalization layer 
65. the learning efficiency in the 2D-K layer 64 will be 
sufficiently improved. Addition of the 2D-K layer 64 con- 
tributes to a surprising number of information process- 
ing capabilities for unknown data as well as a 
remarkably improved convergence efficiency on learn- 
ing. 

The neural network having the 2D-K layer can be 
completed by means of expanding the above mentioned 
back propagation method so that the learning proce- 
dure can be determined in a similar manner as in the 
back propagation method. 

The self-organization requires that the system 
uses, during adaptation of initial several thousands 
times, no other information other than the incoming pat- 
terns and no data are fed back from the output layer. 
After completion of self -organization the network is 
trained according to the back propagation algorithm. 
The neural network having a structure according to this 
embodiment can be trained with less data for a shorter 
period of training cycle. 

In the above mentioned second embodiment, the 
neural network already trained can be coded by using a 
programming language such as C-language. The net- 
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work may be used as an imperative application system 
or packaged as a control microprocessor. In this event, 
the network can be memorized in a read only memory 
for every one type of commercial vehicles. 

For the portion of algorithm that is established read- 
ily in logical, a well-known expert system may be 
applied to achieve a prediction system using a combina- 
tion of logic circuit for the neural network and the expert 
system. 

While the above embodiments have thus been 
described in conjunction with automatic crash evasive 
operation, it is possible to give precedence to the 
driver's operation. For example, it is possible to issue 
appropriate warnings to the driver before actuation of 
the safety drive ensuring arrangement 50. For this pur- 
pose, an audible signal such as an alarm sound may be 
generated to alert the driver to potential hazards. 

Claims 

1. An apparatus for predicting and evading potential 
crash of a vehicle and predicting real crash of a 
vehicle and acting correspondingly, comprising: 

image pick-up means mounted on the vehicle 
for picking up images of actual ever-changing 
views when the vehicle is on running to pro- 
duce actual image data; 
crash predicting means comprising a neural 
network associated with said image pick-up 
means, said crash predicting means succes- 
sively supplied with the actual image data for 
predicting potential occurence of crash 
between the vehicle and potentially dangerous 
objects to produce an operational signal when 
there is possibility of crash, 
safety drive ensuring means connected to said 
crash predicting means for actuating, in 
response to the operational signal, predicting 
potential or real crash, 
characterized in that 

an occupant protecting mechanism including 
an airbag is provided, 

said neural network being adapted and trained 
to predict real occurence of a crash and being 
connected with said airbag. to release said air- 
bag in case of real crash, 
a safety drive ensuring arrangement is pro- 
vided, 

said neural network being adapted and trained 
to predict occurence of potential crash and 
being connected with said safety drive ensur- 
ing arrangement to perform proper function to 
evade crash, 

said neural network therefore being trained 
with training data to predict the possibility of 
potential occurence and real occurence of 
crash, the training data representing ever- 



changing views previously picked up during 
driving of the vehicle and after real crash. 

2. Apparatus according to claim 1 , 
5 characterized in that 

means for producing an audible signal to alert 
the driver to potential occurence of crash are 
provided. 

10 

3. Apparatus according to claim 1 , 
characterized in that 

said neural network comprises at least an input 
is layer and an output layer, and 

training data are supplied to the input layer 
while the output layer is supplied with, as 
teacher data, 

flags representing "crash expected" and "no 
20 crash exprected", respectively, of the vehicle. 

4. Apparatus according to claim 3, 
characterized in that 

25 said neural network comprises a two-dimen- 

sional self-organizing competitive learning 
layer as an intermediate layer. 

5. Apparatus according to claim 1 , 
30 characterized in that 

said neural network is coded after completion 
of learning and implemented in the vehicle. 

35 6. Apparatus according to claim 1 , 
characterized in that 

said safetey drive ensuring means includes a 
steering actuator and the occupant protecting 
40 mechanism includes a steering system of the 

vehicle. 

7. Apparatus according to claim 1 , 
characterized in that 

45 

said safety drive ensuring means includes a 
throttle actuator and the occupant protecting 
mechanism includes a throttle system of the 
vehicle. 

50 

8. Apparatus according to claim 1 , 
characterized in that 

said safety drive ensuring means includes a 
55 brake actuator and the occupant protecting 

mechanism includes a brake system of the 
vehicle. 
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9. Method for predicting, and evading potential crash 
of a vehicle and for predicting real crash of a vehicle 
and acting correspondingly, comprising: 

image pick-up means mounted on the vehicle s 
for picking up images of actual ever-changing 
views when the vehicle is on running to pro- 
duce actual image data; 
crash predicting means comprising a neural 
network associated with said image pick-up io 
means, said crash predicting means succes- 
sively supplied with the actual image data for 
predicting potential occurence of crash 
between the vehicle and potentially dangerous 
objects to produce an operational signal when is 
there is possibility of crash, 
safety drive ensuring means, connected to said 
crash predicting means for actuating, in 
response to the operational signal, the occu- 
pant protecting mechanism including an air- 20 
bag, 

and a safety drive ensuring arrangement to 
perform proper function to evade crash, which 
are operatively connected thereto and 
equipped in the vehicle, 25 
whereby 

said neural network contains previously taken 
image data formed of successive scenes for 
causing accidents and is trained by a back 
propagation method for realizing conditions of 30 
causing said accidents, whereby the training of 
said neural network is performed by supplying 
training data to an input layer of said neural net- 
work while an output layer of said neural net- 
work is supplied with, as teacher data, flags 35 
representing "crash expected" and "no crash 
expected", 

respectively, of the vehicle, 
said neural network evaluates the actual image 
data by itself, ' <o 
said neural network judges if the vehicle col- 
lides with the objects noticed in the actual 
image data of the image pick-up means, and 
outputs said operational signal in case of pre- 
diction of occurence of a crash with said object. 45 

10. Method according to claim 9, 
characterized in that 

said neural network comprises a two-dimen- so 
sional self-organizing competitive learning 
layer as an intermediate layer. 



1 1 . Method according to claim 9, 
characterized in that 

said neural network is coded after completion 
of learning and implemented in the vehicle. 
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Patentanspruche 

1. Vorrichtung zum Vorhersagen und Vermeiden 
eines potentieilen ZusammenstoBes eines Fahr- 
zeugs und zum Vorhersagen eines realen Zusam- 
menstoBes eines Fahrzeugs und zum 
entsprechenden Agieren, mit: 

einer Bildaufnahmeeinrichtung, die auf dem 
Fahrzeug angebracht ist, um Bilder von aktuel- 
len, standig wechselnden Ansichten aufzuneh- 
men, wenn das Fahrzeug fahrt, um aktuelle 
Bilddaten zu erzeugen; 

einer ZusammenstoB-Vorhersageeinrichtung 
mit einem neuronalen Netzwerk, das der Bild- 
aufnahmeeinrichtung zugeordnet ist, wobei der 
ZusammenstoB-Vorhersageeinrichtung suk- 
zessive die aktuellen Bilddaten zum Vorhersa- 
gen eines potentieilen Auftretens eines 
ZusammenstoBes zwischen dem Fahrzeug 
und potentiell gefahrlichen Objekten zugefOhrt 
sihd, um ein Operationssignal zu erzeugen, 
wenn die Moglichkeit eines ZusammenstoBes 
besteht, 

einer Einrichtung zur Gewahrleistung der Fahr- 
sicherheit, die mit der ZusammenstoB- Vorher- 
sageeinrichtung verbunden ist, um in Reaktion 
auf das Operationssignal, das einen potentiei- 
len Oder realen ZusammenstoB. vorhersagt, 
betatigt zu werden, 
dadurch gekennzeichnet, daB 
ein Insassenschutzmechanismus mit einem 
Airbag vorgesehen ist, 

das neuronale Netzwerk zur Vorhersage eines 
realen Auftretens eines ZusammenstoBes 
angepaBt und trainiert ist und mit dem Airbag 
verbunden ist, um den Airbag im Fall eines rea- 
len ZusammenstoBes auszulOsen, 
eine Anordnung zur Gewahrleistung der Fahr- 
sicherheit vorgesehen ist, 
das neuronale Netzwerk zur Vorhersage des 
Auftretens eines potentieilen Zusammensto- 
Bes angepaBt und trainiert ist und mit der 
Anordnung zur Gewahrleistung der Fahrsicher- 
heit verbunden ist, um eine geeignete Funktion 
zur Vermeidung eines ZusammenstoBes 
durchzufuhren, 

das neuronale Netzwerk daher mit Trainings- 
daten trainiert ist, um die Moglichkeit des 
potentieilen Auftretens und des realen Auftre- 
tens eines ZusammenstoBes vbrherzusagen, 
wobei die Trainingsdaten standig wechselnde, 
fruher aufgenommene Ansichten darstellen, 
die wahrend des Fahrens eines Fahrzeug es 
und nach einem realen ZusammenstoB aufge- 
nommen sind. 

2. Vorrichtung nach Anspruch 1 , 
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dadurch gekennzeichnet, 

daB eine Einrichtung zur Erzeugung eines hor- 
baren Signals vorgesehen ist, um den Fahrer 
vor dem potentiellen Auftreten ernes Zusam- s 
menstoBes zu warnen. 

Vorrichtung nach Anspruch 1 , 
dadurch gekennzeichnet, 



daB das neuronale Netzwerk mindestens eine 
Eingabeschicht und eine Ausgabeschicht auf- 
weist, und Trainingsdaten der Eingabeschicht 
zugefQhrt "werden, wahrend der Ausgabe- 
schicht als Lerndaten Kennzeichen zugefQhrt 
werden, die "ZusammenstoB erwartet" bezie- 
hungsweise "kein ZusammenstoB erwartet" 
des Fahrzeugs reprasentiereh. 
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4. Vorrichtung nach Anspruch 3, 
dadurch gekennzeichnet, 



daB das neuronale Netzwerk eine zweidimen- 
siohale, selbstbrganisierende kompetitive 
Lernschicht als Zwischenschicht aufweist. 25 

Vorrichtung nach Anspruch 1 , 
dadurch gekennzeichnet, 

daB das neuronale Netzwerk nach AbschluB 30 
des Lernens kodiert und in dem Fahrzeug 
implementiert ist. 



Vorrichtung nach Anspruch 1 , 
dadurch gekennzeichnet, 



8. 



35 



daB die Einrichtung zur Gewahrieistung der 
Fahrsicherheit eine Steuerungsbetatigung auf- 
weist und der Insassenschutzmechanismus 
ein Steuerungssystem des Fahrzeugs auf- 40 
weist 

Vorrichtung nach Anspruch 1 , 
dadurch gekennzeichnet, 



daB die Einrichtung zur Gewahrieistung der 
Fahrsicherheit eine Drosselklappenbetatigung 
aufweist und der Insassenschutzmechanismus 
ein DrosselWappensystem des Fahrzeugs auf- 
weist. 

Vorrichtung nach Anspruch 1 , 
dadurch gekennzeichnet, 

daB die Einrichtung zur Gewahrieistung der 
Fahrsicherheit eine Bremsenbetatigung auf- 
weist und der Insassenschutzmechanismus 
ein Bremsensystem des Fahrzeugs aufweist. 
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9. Verfahren zum Vorhersagen und Vermeiden eines 
potentiellen ZusammenstoBes eines Fahrzeugs 
und zum Vorhersagen eines realen Zusammensto- 
Bes eines Fahrzeugs und zum entsprechenden 
Agieren, mit: 

einer Bildaufnahmeeinrichtung, die auf dem 
Fahrzeug zur Aufnahme von Bildern von aktu- 
ellen, standig wechselnden Ansichten ange- 
bracht ist, wenn das Fahrzeug fahrt, um 
aktuelle Bilddaten zu erzeugen; 
einer ZusammenstoB-Vorhersageeinrichtung 
rn'rt einem neuronalen Netzwerk, das der Bild- 
aufnahmeeinrichtung zugeordnet ist, wobei der 
ZusammenstoB-Vorhersageeinrichtung suk- 
zessive die aktuellen Bilddaten zur Vorhersage 
des potentiellen Auftretens eines Zusammen- 
stoBes zwischen dem Fahrzeug und potentiell 
gefahrlichen Objekten zugefQhrt werden, um 
ein Operationssignal zu erzeugen, wenn die 
MOglichkeit eines ZusammenstoBes besteht, 
einer Einrichtung zur Gewahrieistung der Fahr- 
sicherheit, die mit der ZusammenstoB-Vorher- 
sageeinrichtung verbunden ist, um in Reaktion 
auf ein Operationssignal betatigt zu werden, 
wobei der Insassenschutzmechanismus einen 
Airbag aufweist, und 

einer Anordnung zur Gewahrieistung der Fahr- 
sicherheit. um eine. geeignete Funktion zum 
Vermeiden eines ZusammenstoBes durchzu- 
fflhren, die bperativ damit verbunden und in 
dem Fahrzeug vorgesehen ist, wobei das neu- 
ronale Netzwerk fruher aufgenommene Bildda- 
ten enthalt, die von aufeinanderfolgenden 
Szenen beim Verursachen eines Unfalls gebil- 
det werden und durch ein Ruck-Propagation- 
Verfahren zum Realisieren von Verhaltnissen, 
die tlnfalle verursachen, trainiert ist, wobei das 
Training des neuronalen Netzwerks durch 
Zufuhren von Trainingsdaten zu einer Eingabe- 
schicht des neuronalen Netzwerks durchge- 
fuhrt wird, wahrend eine Ausgabeschicht des 
neuronalen Netzwerks mit Kennzeichen als 
Lerndaten versorgt wird, die "ZusammenstoB 
erwartet" und "kein ZusammenstoB erwartet" 
des Fahrzeugs reprasentieren, 
das neuronale Netzwerk die aktuellen Bildda- 
ten selbst bewertet, 

das neuronale Netzwerk beurteilt, ob das Fahr- 
zeug mit den beobachteten Objekten in den 
aktuellen Bilddaten der Bildaufnahmeeinrich- 
tung kollidiert und das Operationssignal im 
Falle einer Vorhersage des Auftretens eines 
ZusammenstoBes mit dem Objekt ausgibt 

1 0. Verfahren nach Anspruch 9, 
dadurch gekennzeichnet, 
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da 3 das neuronale Netzwerk eine zweidimen- 
sionale, selbstorganisierende kompetrtrve 
Lernschicht als Zwischenschicht aufweist 

11. Verfahren nach Anspruch 9, s 
dadurch gekennzeichnet, 

daB das neuronale Netzwerk nach Vollendung 
des Lernens kodiert und in dem Fahrzeug 
implementiert wird. to 

Revendications 

1. Appareil pour predire et prevenir une collision 
potentielle d'un vehicule et pour predire une colli- is 
sion reeile d'un vehicule et agir en consequence, 
comprenant: 

un moyen de saisie d'image monte sur le vehi- 
cule pour saisir, quand le vehicule roule, des 20 
images de vues reelles constamment chan- 
geantes, afin de produire des donnees d'irria- 
ges reelles; 

un moyen de prediction de collision compre- 
nant un reseau neuronal associe audit moyen 2s 
de saisie d'image, (edit moyen de prediction de 
collision 6tant alimerite successivement avec 
les donnees d 'images pour predire la survenue 
potentielle d'une collision entre le veJiicule et 
des objets potentiellement dangereux afin de 30 
produire un signal d'op6ration lorsqu'il existe 
un risque de collision; 

un moyen assurant une conduite sure, relie au 
moyen de prediction de collision pour un 
actionnemerrt, en reponse au signal d*op6ra- 35 
tion, et pour une prediction de collision poten- 
tielle ou reelle. 

caracterise en ce que: 
un mecanisme de protection d'occupant com- 
prenant un coussin gonflable est present, 40 
ledit reseau neuronal etant adapte et eduque 
de maniere a predire la survenue reelle d'une 
collision et etant relie audit coussin gonflable 
pour declencher le fonctionnement dudit cous- 
sin gonflable en cas de collision reelle, 45 
un agencement assurant une conduite sure est 
prevu, ledit reseau neuronal etant adapte et 
eduque de maniere a predire la survenue d'une 
collision potentielle et etant relie audit agence- 
ment assurant une conduite sure afin de rem- so 
plir une fonction appropriee pour prevenir une 
collision, 

ledit reseau neuronal etant par consequent 
eduque avec des donnees d'education de 
maniere a predire le risque de la survenue ss 
potentielle et de la survenue reelle d'une colli- 
sion, les donnees d'education representant 
des vues constamment changeantes saisies 
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anterieurement pendant que le vehicule roule 
et apres une collision reelle. 

2. Appareil selon la revendication 1 , caracterise en ce 
qu'un moyen servant a produire un signal audible 
pour avertir le conducteur d'une survenue poten- 
tielle de collision est present. 

3. Appareil selon la revendication 1 , caracterise en ce 
que ledit reseau neuronal comprend au moins une 
couche d'entree et une couche de sortie, et que les 
donnees d'education sont fournies a la couche 
d'entree pendant que la couche de sortie regoit, en 
tant que donnees d'educateur, des indicateurs 
representant une "collision prevue*' et "aucune col- 
lision prevue" du v6hicule, respectivement. 

4. Appareil selon la revendication 3, caracterise en ce 
que le reseau neuronal comprend, en tant que cou- 
che ihtermediaire, une couche bidimensionnelle 
d'apprentissage competitrif s'auto-organisant 

5. Appareil selon la revendication 1 , caracterise en ce 
que ledit reseau neuronal est code apres acheve^ 
ment de I'enseignement et est integre dans le vehi- 
cule. 

6. Appareil selon la revendication 1 , caracterise en ce 
que ledit moyen assurant une conduite sure com- 
prend un actionneur de direction et le mecanisme 
de protection d'occupant comprend le systeme de 
direction du vehicule. 

7. Appareil selon la revendication 1 , caracterise en ce 
que ledit moyen assurant une conduite sure com- 
prend un actionneur de papillon des gaz et le meca- 
nisme de protection d'occupant comprend le 
systeme de papillon des gaz du vehicule. 

8. Appareil selon la revendication 1 , caracterise en ce 
que le moyen assurant une conduite sure com- 
prend un actionneur de freins et le mecanisme de 
protection d'occupant comprend le systeme de 
freins du veiiicule. 

9. Procede pour predire et prevenir une collision por- 
tentielle d'un vehicule et pour predire une collision 
reelle d'un vehicule et agir en consequence, com- 
prenant: 

un moyen de saisie d'image monte sur le vehi- 
cule pour saisir, quand le vehicule roule, des 
images de vues reelles constamment chan- 
geantes afin de produire des donnees dlma- 
ges reelles; 

un moyen de prediction de collision compre- 
nant un reseau neuronal associe audit moyen 
de saisie dlmages, ledit moyen de prediction 
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de collision recevant de fagon successive les 
donnees d'images reelles pour predtre la sur- 
yehue potentielle d'une collision, entre le v6hi- 
cuie et des objets potentiellement dangereux 
afin de produire un signal d'operation quand il 5 
existe un risque de collision,: 
un moyen assurant une conduite sure, relie 
audit moyen de prediction de collision pour 
: actionner, en reponse au signal d'operation, le 
mecanisme de protection comprenant un cous- 10 
sin gonflable.el un agencement asurant une 
conduite sure pour remplir une fbnctibn appro- 
priee afin de prevenir une collision, qui y sont 
relies fonctionnellement et montes dans le 
vehicule, is 
grace a quoi (edit reseau neuronal contient des 
donnees d'images saisies anterieurement, for- 
mees de scenes successives entratnant des 
accidents et est eduqu6 par un procede de 
retro-propagation servant a realiser les condi- 20 
tions entratnant lesdits accidents, I'education 
dudit reseau neuronal etant effectuee par four- 
niture de donnees d'education a une couche 
d'entree dudit reseau neuronal pendant qu'une 
couche de sortie dudit reseau neuronal recoit, 25 
en tant que donnees d'educateur, des indica- 
teurs representant une "collision prevue" et 
"aucune collision prevue" du vehicule, respecti- 
vement, 

I edit reseau neuronal evalue par lui-meme les 30 
donnees d'images reelles, 
ledrt reseau neuronal juge si le vehicule heurte 
les objets mentionnes dans les donnees d'ima- 
ges reelles du moyen de saisie d'images et 
emet ledit signal d'operation dans le cas d'une 35 
prediction de survenue d'une collision avec 
ledit objet 

1 0. Procede selon la revendication 9, caracterise en ce 
que ledit reseau neuronal comprend, en tant que 40 
couche intermediate, une couche bidimensionnelle 
d'apprentissage competitif s'auto-organisant. 

1 1 . Procede selon la revendication 9, caracterise en ce 
que ledit reseau neuronal est code apres acheve- 45 
ment de I'enseignement et est integre dans le vehi- 
cule. 
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